Diagnostic accuracy of Truenat MTB Plus, Truenat MTB Ultima and Xpert MTB/RIF Ultra for the diagnosis of pulmonary TB in an HIV-endemic setting

Background: Truenat MTB Plus (MTB Plus) and MTB Ultima (Ultima) are World Health Organization-endorsed low-complexity tuberculosis (TB) tests, however, performance data are scarce. Methods: Adults (≥18 years; n=498) self-presenting with symptoms to primary care clinics in Cape Town, South Africa (19/02/2016–22/02/2023) provided sputa. We evaluated the accuracy of MTB Plus and Ultima, with Xpert MTB/RIF Ultra (Ultra) as a comparator, vs. a single culture (TB reference standard) or MTBDRplus on an isolate (rifampicin susceptibility reference standard). Results: The proportion of MTB Plus and Ultima unsuccessful results was 20% (95% confidence interval 17, 23) and 14 (11, 16), respectively, with ≥half resolving upon retesting the same eluate. In a three-way analysis, MTB Plus, Ultima and Ultra had sensitivities of 84% (78, 88), 90% (85, 93), and 92% (87, 95), and specificities of 95% (92, 97), 85% (80, 88) and 95% (92, 97) for TB. The proportion of unsuccessful results for MTB-RIF Dx done the same day as DNA extraction was 9% (3, 16; MTB Plus-positives) and 18% (10, 26; Ultima-positives) [if after day-of-extraction, these were 27% (18, 35) and 44% (35, 51)]. Same-day rifampicin susceptibility testing was often unsuccessful in samples with “very low” load [73% (58, 89) MTB Plus, 75% (65, 86) Ultima] but had 100% (40, 100) sensitivity and 99% (96, 100) specificity (for both MTB Plus- or Ultima-positive DNA). Lot variation in unsuccessful and false-positive results was observed. Conclusion: Ultima showed comparable sensitivity to Ultra but specificity, lot variation, and, like MTB-RIF Dx, unsuccessful result rates were suboptimal. Funding: European & Developing Countries Clinical Trials Partnership, and South African Medical Research Council.


Introduction
Rapid diagnostics and treatment can reduce global tuberculosis (TB) incidence (1).In 2022, ~30% of the 10.6 million new TB cases were undiagnosed, driving transmission (1).Better diagnostics, including cheaper tests, are key to addressing this care cascade gap.
Xpert MTB/RIF Ultra (Ultra; Cepheid, Sunnyvale, USA), a WHO-recommended con rmatory nucleic acid ampli cation test (NAAT) for Mycobacterium tuberculosis complex (MTBC), targets rpoB, IS6110 and IS1081, with 91% sensitivity in people with presumptive TB (78% in smear-negative people) (2).Ultra has, due to free DNA associated with non-intact cells, suboptimal speci city in people with previous TB (3,4), which has resulted in countries changing their diagnostic algorithms (5).Alternative tests may be less affected by old DNA or DNA associated with non-intact cells.Furthermore, Ultra and its required equipment is, even with concessional pricing, nancially out-of-reach for many TB programmes (6, 7).This, compounded with stock outs and overreliance on a single manufacturer, motivated the development of fast-follower technologies.
Truenat MTB Plus (MTB Plus; Goa, India) is a low-complexity NAAT that utilises chip-based real-time PCR for TB detection by amplifying nrdZ and IS6110 (8).It involves a two-step process where DNA extraction is rst done using the Trueprep instrument, DNA is manually transferred into a PCR tube then placed into a separate Truelab instrument.Truenat devices are portable and battery-operated (9).MTB Plus was WHO-endorsed in 2020 for sputum, however, it was noted more data were needed especially from people living with HIV (PLHIV), for whom data was extrapolated from other sources (10).A recent study including hospitalised PLHIV reported 85% MTB Plus sensitivity (55% in smear-negatives) (11).
Molbio recently developed the Truenat MTB Ultima (Ultima), which additionally includes IS1081 (12).Ultima is not WHO endorsed but is designed to provide superior sensitivity to existing products (13).There are published performance data of Ultima only on tongue swabs, where it had 71% sensitivity and 97% speci city (Ultra had 76% sensitivity and 100% speci city) (14).
Truenat MTB-RIF Dx (MTB-RIF Dx) is a re ex test to amplify rpoB in the same sputum DNA eluate detected as TB positive by MTB Plus or Ultima.MTB-RIF Dx is WHO-endorsed with 84% sensitivity and 97% speci city when re exed from MTB Plus-positive DNA (10).DNA detected as positive by Ultima, which may not have been detected by MTB Plus, may result in differences in MTB-RIF Dx performance.
Our objective was to therefore assess, in our high burden country of South Africa where HIV is frequent and many people have previous TB, the accuracy of Ultima on sputum for the diagnosis of pulmonary TB in people presenting to care, with MTB Plus and Ultra as comparators.We also assessed MTB-RIF Dx in people diagnosed with TB.We hypothesised that Ultima would have similar performance to Ultra.

Ethics
The study was conducted in accordance with the Declaration of Helsinki and was approved by Stellenbosch University Faculty of Health Sciences Research Ethics Committee (N14/10/136) and the City of Cape Town (6470).People provided a written informed consent.

Study design, participant ow and specimen collection
Adults (≥18 years) self-presenting with presumptive TB (15), not on treatment currently or within the last two months were consecutively recruited between 19 February 2016 and 22 February 2023 at primary care clinics in Cape Town, South Africa.Demographic, clinical, and microbiological data were captured on REDCap (16).Previous TB status was obtained via questionnaire.Three sputa specimens were provided over two consecutive workdays (Figure 1).Participants unable to expectorate were induced as described (17).To enable head-to-head comparisons with MTB Plus and Ultima tests, only sputa from people with successful Ultra and culture results (positive or negative) were selected.

Smear microscopy and culture
Each participant provided at least three sputum samples.The most viscous (visually determined) sputum was used for double Ziehl-Neelsen smear microscopy and a MGIT960 liquid culture (BD, Franklin Lakes, USA).The remaining two samples were arbitrarily selected for Ultra (done fresh) or Truenat tests (done after biobanking involving storage at -80°C) (Figure 1).GenoType MTBDRplus (v2.0;Bruker-Hain Life Sciences, Nehren, Germany) was done on culture-positive growth for MTBC and rifampicin resistance detection.
Truenat MTB Plus, Ultima, and MTB-RIF Dx Truenat tests (Molbio, Goa, India) were done between 01/11/ 2022-05/06/2023 per the manufacturer's instructions (12,19,20).In brief, 0.5 mL of thawed sputum was added to 2.5 mL specimen reagent (lysis buffer).The mixture (3 ml) was loaded on the Trueprep (v2) instrument for DNA extraction, resulting in ≥100 μl eluate.Immediately upon extraction, 6 μl eluate was used for each MTB Plus and Ultima.A test was repeated on the eluate if that test's rst result was unsuccessful (invalid or error).MTB-RIF Dx was performed, either after biobanking and thawing (n=127) or same day [n=80; after an update to the instructions for use were issued (21)], on MTB Plus-or Ultima-positive DNA using the same eluate.No repeat testing was done for MTB-RIF Dx unsuccessful eluates.Two percent (9/504) of people were erroneously double tested using MTB Plus (n=4) and Ultima (n=5) and, for each test, one result from each person was randomly selected.Clinical information and reference standard results were unavailable to test operators.The MTB Plus, Ultima, and MTB-RIF Dx test results were categorized as successful (MTBC detected or not detected, and RIF resistance detected or not detected, respectively) or unsuccessful (invalid, error or indeterminate, on Truelab).

De nitions Microbiological Reference Standard (MRS)
A de nite TB case was de ned as MGIT960 culture-positive sputum (with MTBC con rmation).A non-TB participant was de ned as culture-negative.
Participants were unclassi able if sputum culture was either contaminated or positive without MTBC speciation.The extended microbiological reference standard (eMRS) includes Ultra.More information and other de nitions are in the supplement.
Statistical analysis STARD guidelines were followed (supplementary pg.17) (22).Sensitivity, speci city, and predictive values of MTB Plus, Ultima, and Ultra were calculated using 2×2 tables versus the MRS or eMRS.All participants had each test attempted (MTB Plus, Ultima, Ultra) and all comparisons involving these tests are therefore head-to-head.We present index test results without retesting unless speci ed otherwise.Diagnostic yield metrics (DYT, diagnostic yield in those tested; DYD, diagnostic yield in those diagnosed) were calculated as described (23).We analysed data using the χ 2 test (including McNemar's test) and the two-sample proportion test (24).We used Stata (v18; StataCorp LLC, USA) and GraphPad Prism (v8; GraphPad Software, USA).
Cases potentially missed in a hypothetical population TB: Applying the overall unsuccessful and false negative rates seen in this study, in 1000 smear-negative cases, MTB Plus and Ultima would each miss 200 cases (MTB Plus: 170 unsuccessful, 30 false-negatives; Ultima: 100 unsuccessful, 100 false-negatives).With repeat testing of those initially unsuccessfully, missed cases would reduce to 100 (70 still unsuccessful, 30 false-negatives) for MTB Plus and 151 (50 unsuccessful, 101 false-negatives) for Ultima.
Rifampicin resistance: Assuming the 1000 smear-negative cases had rifampicin resistance and immediate rifampicin testing done using MTB-RIF Dx, of the 800 correctly diagnosed as TB-positive by MTB-Plus (without repeat of unsuccessful results), 72 would be missed (all MTB-RIF Dx unsuccessful, assuming MTB-RIF Dx had 100% sensitivity for rifampicin resistance).For MTB-RIF Dx re exed from Ultima-positive, 144 would be missed (all MTB-RIF Dx unsuccessful, assuming MTB-RIF Dx had 100% sensitivity for rifampicin resistance).However, patients with those phenotypic characteristics might experience different unsuccessful and false negative rates.

Discussion
We evaluated the accuracy of Ultima in comparison to MTB Plus and Ultra on sputum from people with symptoms of TB in a high HIV and TB burden setting.
Our key ndings are: 1) the proportion of unsuccessful results are signi cant (~10% with Ultima) and result in missed TB diagnoses, however, retesting halves the number of participants who do not receive a result, 2) Ultima sensitivity was similar to that of MTB Plus and Ultra, 3) Ultima speci city was low (~85%), resulting in approximately 1 in 5 positive results being false-positive (not associated with previous TB), 4) lot variation in MTB Plus and Ultima performance was observed, and 5) MTB-RIF Dx must be done immediately on eluted DNA, has approximately double (19%) the unsuccessful rate on Ultimapositive rather than MTB Plus-positive DNA (even when done fresh) and, due to a low probability of success, should not be done on samples on samples with an MTB Plus or Ultima semi-quantitation classi cation of "very low".Together, these data have implications for Truenat adoption.
There are, to our knowledge, no published data of Ultima's accuracy on sputum.In addition to Ultima, our study increases the evidence base for MTB Plus, in whom data from PLHIV and people with a history of TB was scarce (8, 11).Although we pre-selected specimens from participants with successful Ultra and culture results, a noteworthy proportion of unsuccessful results prior to repeat testing for MTB Plus (17%) and Ultima (10%) were observed consistently over the testing period without any temporal association.This parallels other studies (8,11,25), that reported invalid MTB Plus results from 9-18%.Importantly, we show that more TB cases are missed due to the test being unsuccessful rather than false-negative, highlighting the importance of quantifying unsuccessful results in test evaluations, something recently highlighted in the recently updated WHO TPP, where an acceptable unsuccessful result rate was de ned as 3-5% (26).
Molbio recommends retesting using the same eluate when the initial TB result is unsuccessful (12,19).This is supported by our data as, upon retesting, 60% of MTB Plus eluates initially unsuccessful became successful (54% for Ultima).Repeating would increase people diagnosed who might not otherwise return to give another sputum.Reasons why our retesting of the same eluate had success may be because, at initial testing, the DNA eluate was not su ciently suspended with the lyophilised pellet containing PCR reagents.Molbio recommends allowing this mixture to stand for 30-60 seconds to achieve a clear solution before proceeding (12,19), however, as the only factor that differed upon retesting was time, the manufacturer should consider extending the duration of standing.Before adopting retesting, laboratories would need to factor in cost and workload.MTB Plus and Ultima had 84% and 90% sensitivity compared to 92% for Ultra.Our MTB Plus sensitivity estimate is like others in high HIV-prevalence settings.Among all participants, sensitivity was similar between MTB Plus and Ultima, but Ultima had higher sensitivity than MTB Plus in PLHIV.Ultra sensitivity was higher than MTB Plus for all participants, consistent with previous ndings from Peru (8), but there was no difference among PLHIV.These data address the shortage of MTB Plus and Ultima data in PLHIV.
Ultima had lower speci city compared to Ultra, which has similar ampli cation targets (IS6110, IS1081).This is despite both Ultima and MTB Plus (which did not show low speci city in the same people despite also has a step where the tube is open) being done in parallel at the same time and in the same qualityassured laboratory.Importantly, this nding persisted when Ultima was evaluated against an eMRS that included Ultra.Furthermore, unlike what we described before for Xpert and Ultra (4, 27, 28), diminished speci city was not associated with previous TB.This speci city nding, which translates into low PPV for Ultima even in our high burden setting (more than 3/10 positives false-positive per MRS, 2/10 per eMRS), necessitates further investigation, especially if Ultima is to be applied in settings where pre-test probability of disease is lower.In the only other comparison of Ultra and Ultima (on tongue swabs), Ultima speci city was lower than that of Ultra.
We noted clinically important performance variation for MTB Plus and Ultima associated with lot number, both in terms of unsuccessful results and false positivity.Similar challenges have been reported for the SILVAMP TB-LAM test (FujiLAM; Fuji lm, Tokyo, Japan), which led to the test's postponement (29, 30).Critically, strati cation of performance data by lot is not in TB study guidance (31) nor part of the STARD criteria (22).Our data suggest this is important to incorporate, including in evidence review processes for policy making.Lastly, the variation in Ultima lot performance may be due to the product not yet being commercially available.Tightening of manufacturer quality control processes may be needed.
Our study's primary purpose was not to assess MTB-RIF Dx's sensitivity for rifampicin susceptibility, which requires further evaluation in people with presumed drug-resistant TB, however, we showed that, when MTB-RIF Dx is applied to Ultima-positive rather than MTB Plus-positive eluates, unsuccessful results are more likely (almost all Ultima-positive "very lows" were MTB-RIF Dx unsuccessful).This is likely because such people were positive exclusively based on the ampli cation of the multicopy gene target (IS1081) that MTB Plus (and MTB-RIF Dx) does not include.Lastly, it remains possible that, as for TB detection, MTB-RIF unsuccessful results may partly resolve upon retesting and, although we did not evaluate this, such a strategy would need to factor in elevated risk of unsuccessful results associated with non-same day testing.
This study addresses a critical research gap by evaluating new and existing Truenat tests for TB detection in a cohort with many PLHIV (the largest to date).
Limitations include the use of biobanked samples for Truenat testing.Truenat samples with unsuccessful results were repeated from the same DNA eluate, however, although Molbio recommends repeating the test on a fresh sample, we show repeating from the same DNA eluate is useful (our approach is likely more feasible in situations where specimen re-collection is unfeasible).Although testing was performed in well-resourced research setting with machines calibrated according to the manufacturers' recommendations and, in the case of the Cepheid and Molbio tests, the tests done years apart, we experienced high rates of unsuccessful results for MTB Plus and Ultima even though sputa from people with an unsuccessful Ultra result were excluded.Further monitoring and research into the extent of these unsuccessful results is required, including in different settings.
In summary, Truenat MTB Plus and Ultima are alternative TB sputum test that met WHO's minimum sensitivity threshold for sputum-based tests for culturepositive TB.Ultima has improved sensitivity compared to MTB Plus in PLHIV.However, Ultima's suboptimal speci city, lot variation, and the relatively high proportion of unsuccessful results (also for same-day MTB RIF Dx testing) require careful further investigation.

Declarations
Author GT conceived and designed the study.GT, APN and RW acquired funding.Clinical staff collected samples and curated clinical data.Laboratory analysis was done by AC, NL, ZN, DM, and ZP.SA, AC and GT did analyses and wrote the rst draft.All authors critiqued analyses and revised the manuscript.

Declaration of interests
We received in-kind donations of diagnostic test kits and equipment from Cepheid and Molbio.Cepheid and Molbio had no role in study design or interpretation of results.The authors have no nancial involvement with any organisation or entity with a nancial interest in, or nancial con ict with, the subject matter or materials discussed in the manuscript apart from those disclosed.Funding GT reports funding from the EDCTP2 programme supported by the EU (RIA2018D-2509, PreFIT; RIA2018D-2493, SeroSelectTB; RIA2020I-3305, CAGE-TB) and the National Institutes of Health (D43TW010350, U01AI152087, U54EB027049, and R01AI136894).

Supplementary Files
This a list of supplementary les associated with this preprint.Click to download. DiagnosticaccuracyUltraandTruenat8Sepsupplement.docx Abbreviations: CI, con dence interval; NPV, negative predictive value; PPV, positive predictive value.Bold font represents p values ≤0.05.

Figure 2 Diagnostic
Figure 2 Diagnostic accuracy MTB Plus and Ultima, compared to Ultra, for TB detection versus the MRS and eMRS.The sensitivity of MTB Plus and Ultima did not differ when the MRS and eMRS were used, and Ultima speci city remained suboptimal.Abbreviations: TB, tuberculosis; MRS, microbiological reference standard; eMRS, extended microbiological reference standard.

Figure 3 Speci
Figure 3 Speci city of MTB Plus and Ultima, compared Ultra, for TB strati ed by MRS and eMRS status and previous TB status.Ultima had reduced speci city compared to Ultra and MTB Plus in participants with (n=166) and without previous TB.Abbreviations: TB, tuberculosis; MRS, microbiological reference standard; eMRS, extended microbiological reference standard.

Table 2 .
Diagnostic accuracy of smear microscopy, MTB Plus, Ultima and Ultra versus the MRS, with strati cation by HIV status.MTB Plus had lower sensitivity than whereas was similar.We did not detect sensitivity differences by HIV status.Data are % (95% CI) n/N.
p Within row p-values: *HIV-positive vs. HIV-negative.Within column p-values for people of the same smear status: † vs. MTB Plus ¶ vs. Ultima Abbreviations: CI, con dence interval; NPV, negative predictive value; PPV, positive predictive value.Bold font represents p values ≤0.05.

Table 3 .
Speci city and positive predictive values of MTB Plus, Ultima and Ultra for TB detection in comparison with MRS strati ed by previous TB.Speci city of MTB Plus, Ultima and Ultra did not differ by previous TB status.Data are % (95% CI) and n/N